Software  Engineering  Institute 


Value-Driven  Iterative  and 
Incremental  Development 

Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

Ipek  Ozkaya 
October  28,  2014 


Mellon  University 


©2014  Carnegie  Mellon  University 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1 .  REPORT  DATE  2.  REPORT  TYPE 

28  OCT  2014  N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

Value-Driven  Iterative  and  Incremental  Development 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

Ozkaya  /Ipek 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Software  Engineering  Institute  Carnegie  Mellon  University  Pittsburgh, 

PA  15213 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited. 

13.  SUPPLEMENTARY  NOTES 

The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

'VRSITT?  apt 

1 8 .  NUMBER  1 9a.  NAME  OF 

rtu  D  A  CPC  T?tn  QDrAMQTDT  U  DUDCHM 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  S  AR 

unclassified  unclassified  unclassified 

17 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Copyright  2014  Carnegie  Mellon  University 

This  material  is  based  upon  work  funded  and  supported  by  the  Department  of  Defense  under  Contract  No.  FA8721-05-C- 
0003  with  Carnegie  Mellon  University  for  the  operation  of  the  Software  Engineering  Institute,  a  federally  funded  research 
and  development  center. 

Any  opinions,  findings  and  conclusions  or  recommendations  expressed  in  this  material  are  those  of  the  author(s)  and  do 
not  necessarily  reflect  the  views  of  the  United  States  Department  of  Defense. 

NO  WARRANTY.  THIS  CARNEGIE  MELLON  UNIVERSITY  AND  SOFTWARE  ENGINEERING  INSTITUTE  MATERIAL 
IS  FURNISHED  ON  AN  “AS-IS”  BASIS.  CARNEGIE  MELLON  UNIVERSITY  MAKES  NO  WARRANTIES  OF  ANY  KIND, 
EITHER  EXPRESSED  OR  IMPLIED,  AS  TO  ANY  MATTER  INCLUDING,  BUT  NOT  LIMITED  TO,  WARRANTY  OF 
FITNESS  FOR  PURPOSE  OR  MERCHANTABILITY,  EXCLUSIVITY,  OR  RESULTS  OBTAINED  FROM  USE  OF  THE 
MATERIAL.  CARNEGIE  MELLON  UNIVERSITY  DOES  NOT  MAKE  ANY  WARRANTY  OF  ANY  KIND  WITH  RESPECT 
TO  FREEDOM  FROM  PATENT,  TRADEMARK,  OR  COPYRIGHT  INFRINGEMENT. 

This  material  has  been  approved  for  public  release  and  unlimited  distribution  except  as  restricted  below. 

This  material  may  be  reproduced  in  its  entirety,  without  modification,  and  freely  distributed  in  written  or  electronic  form 
without  requesting  formal  permission.  Permission  is  required  for  any  other  use.  Requests  for  permission  should  be 
directed  to  the  Software  Engineering  Institute  at  permission@sei.cmu.edu. 

DM-0001795 


— _  Fall  2014  SEI  Research  Review 

Software  Engineering  Institute  Carnegie  Mellon  University  Presenter  Last  Name  and  Date  2 

©2014  Carnegie  Mellon  University 


Value-Driven  Incremental  Development 


The  current  approach  in  highly-regulated  domains,  such  as  DoD,  still 
depends  on  lengthy  requirements,  design,  test,  and  evaluation  cycles 

•  Excessive  documentation  without  analysis 

•  Monolithic  architecting,  modeling,  or  assurance  activities  result  in  rework 

The  goal  of  this  project  is  to  develop  architecture  dependency  analysis 
focused  techniques  to  integrate  architecture  analysis  with  development 
efforts  early-on  and  continuously: 

Our  approach  includes: 

•  Architecture  dependency  management 

•  Incremental  assurance  structuring 

•  Quality  attribute  allocation  techniques 
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Technical  Approach 
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1  Application  Layer 


Architectural  Dependencies 


Technical  approach 

•  Track  additional  information  (e.g.,  safety 
critical  testing  level)  using  a  DSM 

•  Extract  fault  ontology  and  propagation- 
related  information  from  architecture 
analysis  tools  (e.g.,  AADL) 

•  Apply  structural  metrics  (e.g.,  stability)  on 
the  augmented  DSM  and  check  on 
collaborator  data 
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Architectural  Dependencies 
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based  analysis 
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Stepper  Motor  Example 

A  stepper  motor  systems  is  an 
open  loop  system  with  no 
feedback  on  the  successful 
execution  of  the  steps  it  must 
take  for  a  position  change 
command. 


•  How  can  we  ensure  that  steps 
are  not  missed  during 
execution? 

•  How  can  we  ensure  that  when 
change  are  made  testing 
resources  are  spent  on  target? 


Software  Engineering  Institute 


Carnegie  Mellon  University 


Fall  2014  SEI  Research  Review 
Presenter  Last  Name  and  Date 

©2014  Carnegie  Mellon  University 


7 


Dependency 


Type  Guide 


Description 


Aggregation 

Data  element  A  and  Data  element  B  have  a  semantic  coherence  that  can 
be  aggregated  as  Module  AB 

Control 

Module  A  depends  on  the  presence  of  a  correct  functioning  module  B. 

Data 

For  a  module  B  to  execute  correctly,  the  syntax  (type  or  format)/semantics 
of  the  data  produced  by  module  A  must  be  consistent  with  the 
assumptions  of  module  B. 

Location 

For  B  to  execute  correctly,  the  runtime  location  of  A  must  be  consistent 
with  the  assumptions  of  B. 

Allocation  of 
responsibilities 

Behavior  and  functionality  assigned  to  design  time  elements,  used  to 
separate  concerns,  e.g.  safety  criticality. 

Sequence  of  flow 

For  B  to  execute  correctly,  it  must  receive  the  data  produced  by  A  in  a 
fixed  sequence  (data  flow). 

For  B  to  execute  correctly,  A  must  have  executed  previously  within  certain 
timing  constraints  (control  flow). 

Physical  resource 
behavior 

For  B  to  execute  correctly,  the  resource  behavior  of  A  must  be  consistent 
with  B’s  assumptions  about  physical  resource  (such  as  bandwidth, 
memory,  storage  capacity,  CPU,  etc.)  usage  or  ownership, 

Quality  of  service 

For  B  to  execute  correctly,  some  property  involving  the  quality  of  the  data 
or  service  provided  by  A  must  be  consistent  with  B’s  assumptions. 

Virtual  resource 
behavior 

For  B  to  execute  correctly,  the  resource  behavior  of  A  must  be  consistent 
with  B’s  assumptions  about  virtual  resource  usage  or  ownership 
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Missing  information 

Module-view  dependencies 


H 

g 

u 

St 

S/1 

E 

IN 

1 

-C 

u 

< 

1 

s/1 

e 

1/1 

SMS- Arch-3 

SMS-Arch-4 

SMS-Arch-S 

<■£> 

1 

u 

■_ 

■it 

j 

i/l 

e 

(/I 

SMS- Arch-7 

SMS- Arch-8 

SMS- Arch- 1:  SMS.SM PCS 

CD 

SMS- Arch-2: 5  MS.5  M PCS .  Desired  Posi  tionState 

D 

SMS-Arch-  3 :  S  MS  .S  M P  CS  .Co  m  ma  nd  e  d  PositiomState 

D 

SMS-Arch-4:  SMS.SM ACT 

CD 

SMS-Arch-5:  SMS.SM ACT.StepsToDo 

D 

SMS-Arch-6:  SMS.SM MOTOR 

SMS-Arch  -  7 :  S  MS  .S  M M  OTO  R .  A  ctu  a  1 P  o  s  iti  o  nState 

D 

SMS- Arch-8:  SMS.SM HM 

Data  and  control 
relationships  can  be  captured 


Multi-view  dependencies 


iH 

■ 

u 

■_ 

■=t 

1 

1/1 

E 

S/1 

IN 

1 

-C 

u 

■_ 

■it 

1 

i/l 

E 

s/l 

m 

_c 

u 

■_ 

■=t 

(A 

E 

S/1 

SMS-Arch-4 

in 

u 

■=t 

a 

S/1 

E 

s/l 

■ 

.s 

u 

< 

1 

1/1 

E 

S/1 

r-. 

■ 

-s 

u 

■_ 

■it 

1 

to 

E 

S/1 

« 

■ 

u 

■_ 

■it 

1 

s/l 

E 

S/1 

SMS-Arch-1:  SMS.SM 

PCS 

CDS 

L 

SMS-Arch-2:  SMS.SM 

PCS.DesiredPositionState 

D 

A 

A 

SMS-Arch-3:  SMS.SM 

PCS.CommandedPo5itionState 

D 

A 

A 

SMS-Arch-4:  SMS.SM 

ACT 

S 

CD 

SMS-Arch-5:  SMS.SM 

ACT.StepsToDo 

D 

SMS-Arch-6:  SMS.SM 

MOTOR 

S 

SMS-Arch-7:  SMS.SM 

MOTOR.ActualPositionState 

A 

A 

D 

SMS-Arch-S:  SMS.SM 

HM 

LP 

Aggregation,  sequence  of 
flow,  location  and  physical 
resource  dependencies  can 
be  captured  when  model- 
based  analysis  is  conducted 


—  _  _  _  Fall  2014  SEI  Research  Review 

Software  Engineering  Institute  Carnegie  Mellon  University  Presenter  Last  Name  and  Date  9 

©  2014  Carnegie  Mellon  University 


Implication  on  testing  resources 


Using  clustering  algorithms  we  can  locate  the  most  connected  areas  that 
need  to  be  tested  further. 
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Incremental  assurance 

Technical  approach 

•  Use  quality  attribute  utility  trees 
and  architectural  dependency 
analysis  to  structure  the 
system’s  architecture  and  its 
assurance  argument 

FY14  results 

•  confidence  map  notation  and 
theory 

•  generation  capability  of 
assurance  cases  from 
requirements 
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Quality  Attribute  Allocation  to  Iterations 


Technical  approach 

•  Use  architecture  tactics-based  and  story 
slicing  techniques  to  link  architectural 
tasks  to  backlog  management  tools 


FY14  results 

•  Patterns  of  iterative  incremental 
development 

•  Rework  occurs  regardless  of  process 
followed 


Ongoing  organization  wide-surveys  of  the 
patterns  of  iterative  incremental  development 
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Example:  Performance  Improvement  Evolution 
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Stimulus 


Enhanced 
“Autopilot”  3x 
feature 


Improved 

- 


Ratcheting 
Response  Measure 


i-urtner 

improved 

order 

capability 


2x 


Customer  submits  orders 
System  processing  trans;  rotary  algorithm;  multi-user 
Process  and  prioritize  transactions 


Ratcheting 

Environment 


n  reature 
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Publications 


Prototypes: 

Semantic  wiki  to  capture  architecture-tactics 
Assurance  case  generation  tool 

Publications: 

Architectural  dependency  analysis  to  understand  rework  costs  for  safety-critical  systems  - 
ICSE2014 

Design  Rule  Spaces:  A  New  Form  of  Architecture  Insight  -  ICSE  2014 

Evolutionary  Improvements  of  Cross-cutting  Concerns:  Performance  in  Practice  -  ICSME 
2014 

Increasing  Confidence  by  Strengthening  an  Inference  in  a  Single  Argument  Leg:  An 
Alternative  to  Multi-Legged  Arguments  -  Dependable  System  Networks  (DSN) 

Using  Al  to  model  quality  attribute  tradeoffs  -  Al  in  Requirements  Engineering  @  RE  2014 

Agile  in  Distress:  Architecture  to  the  Rescue  -  Principles  of  Large-Scale  Agile  Development 
@  XP  Conference 

Research  Workshops  Led: 

6th  International  Workshop  on  Managing  Technical  Debt  @  ICSME  2014 
1st  International  Workshop  on  Software  Architecture  &  Metrics  @  WICSA  2014 
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Going  Forward  in  FY15 


Improving  Software  Sustainability  through  Data-driven  Technical 
Debt  Management 

What  code  and  design  indicators  can  be  discovered  in  a  repeatable  way 
to  measure  and  manage  technical  debt? 


Incremental  Life  Cycle  Assurance  of  Critical  Systems 

How  can  system  assurance  confidence  and  cost  be  improved  through 
requirements  coverage  and  consistency  checking  and  compositional 
verification  evidence? 
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Team:  Value-Driven  Incremental  Development 


SEI  team  members 

•  Ipek  Ozkaya,  PhD  (lead) 

•  Robert  Nord,  PhD  (co-lead) 

•  Stephany  Bellomo,  MSc. 

•  Julien  Delange,  PhD 

•  Neil  Ernst,  PhD 

•  Peter  Feiler,  PhD 

•  Ian  Gorton,  PhD 

•  John  Goodenough,  PhD 

•  Rick  Kazman,  PhD 

•  Ari  Klein,  PhD  Candidate 

•  Chuck  Weinstock,  PhD 


Collaborators 

•  Prof.  Philippe  Kruchten,  PhD 
University  of  British  Columbia 

•  Prof.  Raghu  Sangwan,  PhD 
Penn  State  University 

•  Prof.  David  Garlan,  PhD 
Carnegie  Mellon  University 

•  John  McGregor,  PhD 
Clemson  University 

•  And  other  industry  and  DoD  collaborators 
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Contact  Information 


Ipek  Ozkaya,  PhD 

SSD  SEAP  Architecture  Practices  Initiative 

ozkava@sei.cmu.edu 

+1  412-268-3551 


Web 

www.sei.cmu.edu 

www.sei.cmu.edu/contact.cfm 

www.sei.cmu.edu/architecture/research/ 
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